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EXECUTIVE SUMMARY 


A. PURPOSE 

The purpose of this thesis is to compare and analyze the effects of 
the Janus(A) default terrain database and a Janus(A) modified terrain 
database on a modeled Javelin operational test. It is part of the Model- 
Test-Model (M-T-M) research program sponsored by the U.S. Army Training and 
Doctrine Command (TRADOC), Analysis Command-Monterey (TRAC-MTRY) and the 
Test and Experimentation Command (TEXCOM), Experimentation Center (TEC). 
This is part of an ongoing effort to accredit the M-T-M concept. 

The research for this thesis was accomplished in three phases. First, 
the operational test area was defined and two duplicate Janus(A) terrain 
database files were created. The terrain characteristics of one of the 
files were modified so that the vegetation information per 50 meter grid 
cell more accurately represented that of the actual test site. Next, a 
data analysis comparison was done on the two files to determine if their 
distributions are the same. Lastly, the Javelin test scenarios were run 
uSing both terrain files and the results compared using statistical 


procedures. 


B. BACKGROUND 

With projected budget cuts, fewer defense dollars will be available 
for operational test and evaluation (OT&E) of weapon systems. The M-T-M 
concept integrates operational field tests with simulation models to 
increase the effectiveness and efficiency of operational tests. The 
concept of M-T-M is to use high resolution simulation model(s) before the 
operational test to help design efficient and meaningful test scenarios and 
tactics, and after the field test to expand the evaluation effort to 
include testing other scenarios and tactics. One of the models currently 
in use is Janus(A), a computer-based, two-sided, high resolution, 
stochastic combat simulation model designed for Army training and combat 


development. It uses a digitized terrain database to represent a portion 


vill 


of the real terrain. This database is based on Digital Terrain Elevation 


Data (DTED) Level I data provided by the Defense Mapping Agency. 


Cc. RESULTS 

The results of the analysis indicated that vegetation code 
distributions of the default terrain database and the modified terrain 
database were not the same. Three scenarios, a deliberate night defense, 
a deliberate day defense and a hasty day defense were modeled using each 
of the terrain databases. Data was collected using the detection range of 
an enemy vehicle by the opposing force as the measure of effectiveness. 
The three different scenarios were each run eight times on each terrain 
database. Analysis indicated that there was a statistically significant 
@Gifference between the detection ranges for two of the scenarios. These 
two scenarios were run on terrain that required extensive modification to 
the vegetation codes. No Significant difference was noted in the third 
scenario. Very few modifications of the terrain data were needed in the 


area where this battle was fought. 


D. RECOMMENDATIONS 

The results do not suggest that the Janus(A) model is invalid. 
However, the results do indicate that further improvements to the terrain 
database will further reduce the errors between results based on the 
Simulated terrain and the actual terrain. This will result in better 
simulation of the line of sight, and the effectiveness of a weapon system 
for a given scenario and tactic. Thus, the overall results of the model 


should better reflect the operational test results. 


ix 





I. INTRODUCTION 


BX. RESEARCH OBJECTIVES 

This thesis is part of the Model-Test-Model (M-T-M) research program 
sponsored by the U.S. Army Training and Doctrine Command (TRADOC), Analysis 
Command-Monterey (TRAC-MTRY) and the Test and Experimentation Command 
(TEXCOM), Experimentation Center (TEC). The M-T-M concept integrates 
operational field tests results with simulation models. As part of an 
ongoing effort to accredit the M-T-M concept, this thesis will compare 
Janus(A) model terrain to the actual test site terrain for the Javelin 


antitank weapon system. 


B. BACKGROUND 

With projected budget cuts, fewer defense dollars will be available 
for operational test and evaluation (OT&E) of weapon systems. The M-T-M 
concept integrates operational field tests with simulation models to 
increase the effectiveness and efficiency of operational tests. The 
concept of M-T-M is to use high resolution simulation model(s) before the 
operational test to help design efficient and meaningful test scenarios and 
tactics, and after the field test to expand the evaluation effort to 
include testing other scenarios and tactics. Thus, the operational test 
and evaluation (OT&E) community will work with analysts using simulations 
and models to design operational tests and then use these models to make 
predictions on the employment, manning level, supply rate, etc. of the 
weapon system. One of the models currently in use is Janus(A), a computer- 
based, two-sided, high resolution combat simulation model designed for Army 
training uses and combat development. In October 1990, Mr. Walter W. 
Hollis, Deputy Under Secretary of the Army (Operations Research) tasked TEC 
at Fort Hunter Liggett, California to improve the M-T-M methodology [HOLL 


89}. TEC enlisted TRAC-MTRY to conduct research in support of the M-T-M 


J 


concept. TRAC-MTRY's research efforts are directed toward the validation 
and accreditation of the M-T-M concept using the Janus(A) high resolution 
combat simulation model. 

The model terrain in Janus(A) is a digital representation of the 
actual test site terrain located at Fort Hunter Liggett. Extensive efforts 
have been made to simulate the test site terrain as closely as possible 
because terrain characteristics affect the movement rates and, most 
importantly, the line of sight (LOS) of combat systems. If the combat 
Simulation model terrain accurately represents the test site terrain, one 
can expect amore realistic simulation of the LOS and the effectiveness of 
a weapon system for a given scenario and tactic. Also, the weapon system's 
tactical emplacement can more closely reflect actual field locations which 
affect both range and LOS. Thus, the overall results of the model should 
better reflect the operational test results if the errors between the model 


terrain and the actual terrain are reduced. 


Cr. RESEARCH METHODOLOGY 

The research for this thesis was accomplished in three phases. First, 
the operational test area was defined and two duplicate Janus(A) terrain 
database files were created. The vegetation codes in one of the files were 
modified to more accurately represent the actual test site terrain. To 
help ensure accuracy, two separate terrain walks were conducted to study 
the vegetation and determine densities for various tree stands from 
different locations in the test site area. Information was also gained 
from an eight meter resolution terrain database extracted from the 
Perspective View Generator and Analysis Systems for Unmanned Sensors 
(PEGASUS) replicator. The PEGASUS terrain database creation system will 
be explained ina later chapter. Next, a data analysis comparison was done 
on the vegetation code distributions from an identical one kilometer area 
in each of the two files to determine their similarities and differences. 


Third, the Javelin test scenarios were run using both terrain files and 


7 


the results compared using statistical procedures. The measure of 
effectiveness (MOE) used to compare the results was the detection range of 


an enemy vehicle by the opposing force. 


D. ORGANIZATION 

This thesis is organized in five chapters. Chapter II describes the 
Javelin and the Model-Test-Model concept. Chapter III contains a 
description of Janus(A), the PEGASUS replicator, the process required to 
build a terrain database and the methods used to perform terrain 
modifications. Chapter IV contains the discussion and results of the 
vegetation code analysis and the Javelin scenario runs. A summary of the 


findings and recommendations is given in the last chapter. 


II. JAVELIN AND THE MODEL-TEST-MODEL CONCEPT 


A. JAVELIN 
The success of the U.S. in the Persian Gulf War has prompted potential 
hostile countries to upgrade their military forces with more advanced 
weapon systems. Thus, the U.S. military can expect to face enemy armored 
vehicles with considerable improvements in lethality, mobility and 
survivability. To meet this new threat, the U.S. has had to continue to 
improve and field new anti-armor weapon systems, capable of defeating these 
new threats. Currently, the U.S. forces rely on three antitank weapon 
systems: the tubular launched optically sighted wire guided missile (TOW), 
the Dragon and the antitank 4 missile (AT4). The TOW is a wire guided 
antitank weapon capable of destroying enemy armored vehicles at up to 3750 
meters. The AT4 is a hand eT EL. fire and forget weapon, capable of killing 
thin skinned armored vehicles at ranges from 65 to 300 meters. The AT4 
replaced the light anti-armor weapon (LAW) to give the infantryman more 
firepower to defeat the new enemy armored personnel carriers. The Dragon 
is a medium antitank wire guided missile capable of destroying enemy armor 
out to 1000 meters. 
However, with the new advances in enemy armor, the U.S. has developed 

a new more effective medium antitank missile, the Javelin, to replace the 
Dragon. 

The Javelin is a top attack, man portable antitank 

missile which gives the infantryman an increased 

capability to engage and defeat threat tanks and other 

armored vehicles. It is a fire and forget weapon system 

which significantly increases gunner’. survivability 

because it no longer requires a gunner to track the 

target for the duration of the missile's flight. [JAVE 

$2) 


The Javelin's greater lethality comes from its ability to attack its 


targets from the top where the armor is the thinnest, its 2000 meter plus 


range and its improved warhead. The Javelin consists of three components, 


the command launch unit (CLU), the launcher and the missile, Figure 1. 
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Figure 1 Javelin Components 


The heart of the Javelin is the CLU. It is an integrated day/night sight 
unit that uses conventional optics or thermal imagery to paint a picture 
of the target to the misile's memory allowing it to track the target on the 
battlefield. The Javelin has a rate of fire of three rounds per minute. 
These improvements compared to the DRAGON's, 1000 meter range, rate of fire 
(two rounds per minute), wire guided missile and inability to attack the 
most vulnerable parts of enemy armored vehicles give the dismounted company 
team greater lethality and increased survivability. Introduction of the 
Javelin to the battlefield will better equip U.S. forces to meet and defeat 


new threat armor technology. 


B. MODEL-TEST-MODEL CONCEPT 
The overall goal of the Model-Test-Model concept (M-T-M) is to improve 


operational test effectiveness. M-T-M accomplishes this by using 


/ 


simulation models from which the operations analyst attempts to predict 
operational test data for a particular test scenario. The simulation model 
provides insights into the feasibility of test scenarios and test 
objectives. This information is used by operational testers at the 
Operational Test and Evaluation Command (OPTEC) to make a more efficient 
test design and by the program manager to help him make more accurate 
decisions affecting the acquisition process. Application of M-T-M includes 
a force on force, free play field test that requires cooperation and 
coordination between testing and analysis/modeling communities. This means 
that the modelers and analysts must be intricately involved with the 
planning and designing of tests and the data collection and data reduction 
techniques. This coordination will ensure that all parties involved 
understand and account for all the peculiarities and problems associated 
with the operational test. 

There are five phases to M-T-M: long term planning, pretest modeling, 
field test, post-test modeling and accreditation. Upon accreditation of 
the model, it can be used by military agencies to conduct careful 


extrapolation to other scenarios. 


a Phase 0 (Long Term Planning) 

Phase 0 begins with the analyst and tester defining 
responsibilities. During this phase, a Memorandum of Agreement (MOA) and 
a Project Coordination Sheet are signed by the organizations involved to 
identify working relationships, specific roles and what type of product is 


expected. 
7 em Phase I (Pretest Modeling) 


This is a modeling phase that relies on high resolution combat 
Simulation models to aid in planning the operational field test. The goal 
is to save time, money and resources in the test design. In this phase, 
the model is prepared and executed with the proposed scenarios. The 
scenarios should incorporate the restrictions developed by the test 


personnel (e.g., size/characteristics of the terrain box, weapon 


/ 


capabilities, rates of advance, force size, etc.). The scenario 
restrictions and the tactics for each scenario are given to the weapon 
system proponent to provide the proponent subject matter experts (SME) a 
doctrinal review of the tactics and test design. After the model is run, 
the test design and tactics are refined as necessary. The results from the 
model are used by the test designers to focus data collection efforts, 
identify external and internal constraints that will have a negative impact 
on the test, and minimize the possibility that test objectives will not be 
accomplished. Phase I continues until the model predicts that the test 


objectives will be met. 


a. Phase II (Field Test) 

During this phase, the operational effectiveness and weapon 
system capabilities are evaluated. The first part of this phase is the 
Initial Operational Test and Evaluation (IOTE). In the IOTE, maneuver 
units perform the successful battle tactics developed by their unit leaders 
in the pretest modeling phase. The tactics are revised as appropriate, 
based on the outcomes of the IOTE. The Operational Test (OT) tactics must 
be unscripted to increase the validity of the OT, but the tactics must 
remain within the test restrictions. It is crucial that the modeler 
understand how the test was conducted and works with the data reduction 
personnel to understand how the data was collected and screened. At the 
conclusion of the field test phase, the modeler obtains the necessary data 


and begins the post-test modeling phase. 


4. Phase III (Post-Test Modeling) 

In this phase, the model is used to provide analysis and feedback 
to test personnel, to explain unexpected test results and to guide the 
modifications of further trials, if necessary. Successive iterations of 
the model are run to calibrate the model’s algorithms and model data. The 
modeler must be careful not to calibrate the model to a specific trial, as 


the goal is to have the model represent the field test within specified 


statistical tolerances. These tolerances are usually defined by the agency 


responsible for accrediting the simulation model. 


S- Phase IV (Validation/Accreditation) 

In phase four, the “calibrated" model is used to extrapolate the 
test results to conditions, scenarios and threats that were not tested due 
to cost, time, or equipment/environmental constraints. This occurs after 
the model is validated and accredited. Validation of a model is the 
process of determining that a model is an accurate representation of the 
intended real-world entity from the perspective of the intended use of the 
model [ARMY 92). Accreditation is performed by OPTEC, based on experience 
and expert judgement that the model is adequate for its intended use [ARMY 
92). It is the certification that the model is acceptable for use for a 
specific type of application [ARMY 92). The credibility of the extended 
results naturally depends on how far an extension is from a calibration 
point (e.g., scenario in jungle terrain extended to desert terrain or 
Mission Oriented Personnel Posture (MOPP) 0 to MOPP 4). Thus, the 
modeler/analyst must take care not to extend the model results to a point 
where the tactical character of the battle changes so much that the, 
calibration representation is no longer valid. 

This thesis will support TRAC-MTRY's ongoing accreditation effort for 
M-T-M using the Janus(A) high resolution combat simulation model. The 
focus will be to compare and analyze the effects of the Janus(A) terrain 
and the actual terrain characteristics on the modeled operational test. 
The detection range of an enemy vehicle by the opposing force will serve 


as the MOE. 


III. TERRAIN SIMULATION PROCESS 


A. JANUS (A) COMBAT MODEL 


1. Overview 
Janus, named for the two-faced Roman god who guards portals, is 
an interactive, computer-based, war-gaming simulation for combat operations 
at the brigade and lower level [JANU 93]. The Janus combat simulation 
model has evolved through several versions, beginning with the Janus(L) 
version that was developed by the Lawrence Livermore National Laboratory 
(LLNL). The U.S. Army TRADOC Analysis Command, White Sands Missile Range, 
New Mexico (TRAC-WSMR) acquired this prototype and developed Janus(T) to 
meet combat development needs. From the Janus(T) version came the Janus 
Army or Janus(A) version which is used for combat development and training 
needs. Janus(A), version 3.0, is the most current version now in use by 
the Army. Today, Janus(A) is developed, maintained and distributed by 
TRAC-WSMR, and is fielded throughout the Army as a tool for trainers and 
analysts in testing, research and combat development. 
Janus(A) 1s a “high resolution, event-driven, two-sided, closed, 
stochastic, ground combat simulation" [JANU 93]. 
e ‘High resolution' means that the smallest object modeled in the 
aes is an individual weapon system (e.g., a soldier and his M16 


« ‘Event driven' means the state of the simulation is updated 
asynchronously when a combat event occurs. 


¢« '‘Two-sided' refers to the two opposing forces, Blue and Red. The Blue 
force and Red force are simultaneously directed on separate monitors 
by two different sets of players. Janus(A) can be used ina single 
player mode as well. 


e ‘Closed' means the disposition of the opposing force is unknown, 
except those locations found by direct intelligence reports from 
friendly forces. 


e ‘Stochastic’ means that certain events are not predetermined, but 
eccur according to the laws of probability. 


« ‘Ground combat' means the principal focus of the simulation is on 
ground maneuver and artillery combat units. 
Janus (A) is also capable of simulating the effects of weather, rotary and 
fixed wing aircraft, day and night visibility, chemical environment, 
minefield employment and other variables. Janus(A) is written entirely in 
the VAX-11 FORTRAN language and currently runs on any Digital Equipment 
Corporation (DEC) VAX family computer using the standard VMS operating 


system. 


2. Janus(A) Battlefield Terrain 
The Janus(A) terrain database is a digitized representation of a 
portion of real world terrain, based on Digital Terrain Elevation Data 
(DTED) Level I data. The DTED Level I data is supplied by the Defense 
Mapping Agency (DMA). The data is converted from a profile plot to a 


contour plot for use in the Janus(A) computer simulation, Figure 2. 
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Profile Plot Contour Plot 


Elevations depicted as profile lInes from Elevations depicted as contour lines 
an oblique perspective. from a vertical perspective. 
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Figure 2 Digital Terrain Elevation Data Level I 





Janus (A) provides the user a computer generated map image of the exercise 
area, showing terrain contours, forested areas, roads, rivers and urban 
areas. Within Janus(A), the user can initialize the size of the exercise 
area based on his needs. The exercise area 1S a Square that is subdivided 
into 360,000 (600 x 600) grid cells. The user can vary the size 
(resolution) of the grid cells from 25m x 25m, 50m x 50m, 100m x 100m to 
200m x 200m. Each grid cell carries the characteristics of the terrain: 
elevation, vegetation or urban area, density and height, road data, rivers 
and bridges within that grid cell. Each characteristic is assigned a 
numeric value that defines the characteristic within that grid cell. For 


example, a numeric value of five for vegetation defines the density and 
, 


al at 


height of the vegetation in that grid cell as a representation of the 
actual tree height and density for that geographic area. All of these 
characteristics are assumed constant throughout the grid cell. By reducing 
the size of the grid cells, the user is able to better represent changes 
in the elevation, vegetation, urban areas, road networks and rivers which 
should better simulate the real world terrain being modeled. This thesis 
will explore this question in Chapter IV by running the Javelin scenarios 
on the generic Janus(A) terrain and on the modified 50 meter resolution 


"real world" terrain (described below). 


B. PEGASUS 


a Overview 

The Perspective View Generator and Analysis Systems for Unmanned 
Sensors (PEGASUS) is a prototype terrain database creation system. It is 
being used by the Army Test and Experimentation Command (TEXCOM), 
Experimentation Center (TEC) at Fort Hunter Liggett, California to produce 
object material descriptors for the PEGASUS real-time perspective view 
generation replicator [BAER 91]. The object material descriptors are the 
information such as height, width, tree canopy shape and other attributes 
that describe various objects such as trees, buildings and vehicles. ‘The 
replicator is to provide substitute video images that are realistic 
representations of the battle maneuvers actually occurring in real time" 
{BAER 91]. The replicator can be perceived as a data flow through a sensor 
response system. 


The system flow starts with photographic measurements of targets, 


cultural features and terrain background. To these are added 
geographic and object size measurements to orient and scale the 
photographic data. The photographic input data is digitized, 


radiometrically calibrated, registered with location and scale 
information and stored on optical disk in a registered image 
database [BAER 91]. 
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oe Terrain Database Creation 
The PEGASUS database creation system is a transputer-based 
computer system, with numerous algorithms designed to input, measure and 
parameterize visual images into an object-oriented database. The 
transputers provide the parallel processing necessary to create the visual 
imagery of this complex database in close to “real-time”. The terrain 


database creation software is depicted in Figure 3. 
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Figure 3 Terrain Database Creation System Diagram 


The diagram shows the variety and flow of information used to create and 
update a PVG database. The system can use the elevation and vegetation 
code data supplied by DMA and the Waterways Experimentation Station (WES) 
respectively, or generate its own initial estimates. The result is an one 
meter resolution terrain database for a 375 square kilometer test area. 


This one meter terrain database can be expanded to a four meter, sixteen 
If 


pipe. 


meter and sixty four meter database by filtering and averaging high 


resolution information. 


Cc. SCENARIO TERRAIN FILE 

The scenario terrain file is the computer simulated terrain that 
represents the actual test site terrain. Within Janus(A) exist the Master 
Terrain Files (MTF) which are based on DMA DTED files. The analyst or 
trainer can customize his specific scenario terrain file using the Janus (A) 
terrain editor which is part of the Janus(A) Database Manager. This 
customizing process determines the modeled test site terrain, and thus 
determines the Lines of Sight (LOS) and movement factors. Each scenario 
terrain file consists of a terrain file (TERAIN###.DAT;#) and a terrain 
screen file (TSCRN###.DAT;#). This section will explain the processes 
used to make the Janus(A) and PEGASUS simulated terrain and discuss the 


role and products of agencies involved in the creation of model terrain. 


s Terrain File (TERAIN##.DAT; #) 
The terrain file contains the digitized data derived from data 


supplied by DMA. 


a. DMA Process 

As discussed earlier, The data supplied by DMA is called the 
Level I Digital Terrain Elevation Data (DTED) which is used for all 
military activities and systems that require landform, slope and elevation 
in a digital format. The DTED Level I data is derived from imagery with 
a one degree by one degree cell size defined by the integer one degree 
latitudes and one degree longitudes of the geographic reference system 
[DIGI 90). The vertical and horizontal datums are Mean Sea Level (MSL) and 
World Geodetic System (WGS) 84, respectively. The elevation data is 
expressed in meters and the information content 1s equivalent to a 
1:250,000 scale resolution. DMA provides this data ina matrix structure 


with an ASCII labeled variable length record format. 
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b. PEGASUS Process 

PEGASUS can use the DTED Level I data supplied by DMA as a 
baseline for the terrain file or generate its own. Using the DTED Level 
I data as a baseline creates no additional error problems because of the 
terrain refinement process in PEGASUS. Within PEGASUS, the Perspective 
View Database (PVDB) details the test scenario terrain area in resolutions 
of one, four, sixteen and sixty four meters and covers a 32.768 x 28.672 
kilometer rectangular area. The coordinates of the rectangle conform to 
the Universal Transverse Mercator (UTM) conventions. The PVDB is organized 
into a Tile/Block/Post (TBP) structure. The PVDB is broken up into an 8 
x 7 collection of tiles, Figure 4, and each tile is partitioned into a 16 
x 16 array of blocks and each block contains an arrangement of data posts, 
Figure 5 [AKIN 89}. The posts contains the elevations, cultural feature 
indicators, gray shade values, surface normal indicators and sun shade 


flags required for the creation of the detailed terrain information. 
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Figure 4 PVDB Tile Structure 
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Figure 5 PVDB Block Structure 


A tile represents an area of ground measuring 4096 x 4096 meters and a 


block represents an area measuring 256 x 256 meters. The following table 
shows the number of data posts ina block, which is dependent on the data 


resolution: 


Table I POSTS WITHIN A BLOCK 





Resolution # of Posts 
64 meter 4 
16 meter 16 


meter 64 
meter 256 


1 


2. Screen File (TSCRN###. DAT; #) 
This file contains data depicting vegetation, grid squares, 


roads, rivers, contour lines and urban areas. 


a. Waterways Experimentation Station (WES) Process 

The Waterways Experimentation Station is part of the Corps 
of Engineers and has a mission of supporting the U. S. Army by means of 
geotechnical support. Terrain database development began with the creation 
of descriptions of basic terrain factors that effect the movement of 
mounted or dismounted units engaged in combat. WES's procedure for 
Geveloping terrain databases involves preparing overlays that describe 
basic terrain factors of the specified test area [BULL 88]. These overlays 
are called primary factor maps. The primary factors associated with each 
basic terrain factor are: land use, slope, soils, obstacles and linear 
features. A listing of the components of the primary factors can be found 
in Appendix A. The primary factor maps are digitized and rasterized at the 
required resolution and combined with the terrain file to form the scenario 


terrain database. 


dD. PEGASUS Process 
Only one process is used to produce the terrain file and 
screen file in PEGASUS. The PVDB contains the cultural feature indicators 
as well as the elevations. This is accomplished through the use of a 32- 
bit terrain element database that provides the information for surface 
objects such as rocks, trees, buildings, etc. The terrain database 
creation diagram, Figure 3, again shows the process for building and 


continuously updating the scenario terrain database. 


D. TERRAIN MODIFICATION 

All direct fire weapons are dependent on lines of sight (LOS) to be 
effective, and Javelin is a direct fire weapon. This simple, but crucial 
fact makes the representation of the actual terrain in the simulation model 
very important. The characteristics of the terrain are a key factor in 


LOS determination, and the LOS is a key factor in the detection of any 
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enemy element. There are two factors that effect LOS on the ground: [CELS 
92] 

e Elevation: The elevation level between forces must allow for LOS. If 
intervening crests mask the LOS between the sensor and the target, no 
direct fire occurs. 

e Vegetation and urban areas: Trees and man-made objects can interfere 
with LOS. So, if vegetation and urban areas exist, they must be 
accurately represented in the model. 

Janus(A) addresses both deterministic and probabilistic aspects of 
detecting a target. It addresses the deterministic aspect by calculating 
whether there is any type of terrain mask between the sensor and the 
target. The probabilistic aspects are addressed by determining if the 
target can be detected through vegetation or urban areas, by considering 
the density of the objects in the line of sight. 

It has already been explained that the density and height of an object 
are assumed constant and uniform throughout a grid cell. However, the 
program modelers of Janus(A) also modeled an area or group of grid cells, 
that has a stand of trees as an area having homogeneous density and tree 
heights. A color picture of this Janus(A) representation for the Javelin 
test area is found in Appendix B. So, to ensure that the LOS for the 
Javelin correctly reflected the actual terrain, this study modified the 
Janus(A) scenario terrain using an eight meter terrain database derived 
from the PEGASUS one meter terrain database. The eight meter terrain 
Gatabase depicts the Fort Hunter Liggett area and was adapted to run ona 
personal computer. Within this database, the user is able to zoom in from 
a top down perspective to see individual trees. Also, when the user clicks 
the mouse on a particular piece of terrain, the military grid coordinates 
along with an elevation are displayed. This allows the user to more 
accurately place vegetation on the Janus(A) simulated terrain. The 
database is now the possession of TRAC-MTRY and the Janus(A) developers at 


TRAC-WSMR are attempting to modify the database program files so the 
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Janus (A) Terrain Research Program (TRP) can read the files into a Janus(A) 
two-dimensional display of the Fort Hunter Liggett area. 

Next, an eight kilometer square area covering the Javelin test site 
area was modeled in Janus(A). Two copies of this scenario terrain were 
stored in separate files. Both files have a resolution of fifty meters. 
Then one of the files was modified using the eight meter PEGASUS database. 
In this scenario file, a vegetation code was placed on, or removed from, 
each fifty meter grid cell and the density of each grid cell was also 
changed, if needed. Subjective judgement was involved in the areas of 
density and tree height and whether or not a grid cell needed trees in it. 
The tree heights can be changed within the terrain editor. However, there 
1s a basic problem with the way the tree heights are handled. The tree 
heights are tied to the density codes. This means that for a density code 
of five a corresponding tree height can be set to eleven meters or any 
other value you pick. The problem is, the tree height then applies for all 
densities of five in the scenario terrain. The model does not take in 
consideration that different types of trees can be found in the same 
scenario area (e.g., deciduous stand of trees and an orange grove) ora 
younger stand of trees may populate a certain sector of the scenario 
terrain. For the test site area, the majority of the trees observed on two 
different terrain walks were approximately nine to thirteen meters high. 
This was verified by forestry sources on the post. There was a problem of 
determining whether trees were needed in a certain grid cell and the 
correct density of trees for a particular grid cell. This was solved be 
looking at the number of trees per fifty meter square area and how close 
the trees were to each other. 

Although, subjective judgement was used to modify Janus(A) terrain, 
the belief is that the modified terrain will better represent the actual 
test site terrain. A color picture of the default and modified terrain can 


be found in Appendix B. 


20 


E. JANUS(A) SUFFICIENCY TO MODEL REAL WORLD TERRAIN 

As stated, Janus(A) is widely used throughout the Army to model 
brigade and lower combat operations. Even though Janus(A) is an approved 
simulation model for the above use, is Janus(A) sufficient to model an 
operational test? This thesis assumes that the Janus(A) model terrain can 
sufficiently represent real world terrain for modeling purposes. 

Currently, there are two methods for modeling terrain. Both of these 
methods have been proven to represent real world terrain adequately. The 
most widely used method is the plateau method. Janus(A) uses the plateau 
method. In this method grid cells encapsulate the terrain characteristics, 
and one elevation value is assigned to each cell. At the respective 
resolution (e.g., 50x50 meter) a system of pilings with flat tops 
(plateaus) are created which represent the real world terrain. This causes 
a stepwise jump or drop transition to each grid cell. Real world terrain 
flows in a more continuous fashion except at cliffs and sinkholes. 
However, as the resolution is increased to twelve meters or six meters the 
plateau method smooths out considerably, depending on the area being 
modeled. This method also lends itself to translation into UTM grid 
coordinates which are the bases for the military grid coordinate system. 

The other method of modeling terrain is called the polygonal method. 
In this method, straight lines are connected to designated elevation 
points. This creates a series of slopes that can be translated into a 
digital contour map. The contour map is then overlayed with a digitized 
screen file with the result that represents real world terrain. However, 
this method does not lend itself to easy translation into the military grid 


coordinate system. 
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IV. ANALYSIS AND DISCUSSION OF JANUS(A) MODEL TERRAIN 


A. GENERAL 

The analysis was conducted in the last two phases. In phase two, the 
vegetation code distributions in the default and modified terrain databases 
were compared to see if the differences were more substantial than would 
be expected because of natural variation. In the third phase, three actual 
IOTE scenarios were simulated using each database to see if the test 
results would be effected. This phase was done because the vegetation 
codes for the terrain files are different. 

For the analysis, a sample one kilometer square was extracted from 
each terrain file. These terrain squares represented the same patch of 
land in each file. Using the Janus(A) terrain utilities, the vegetation 
codes for each one eiaerer es sample were recorded for analysis. in 
Janus(A) the vegetation codes can only take the the value of discrete 
integers from zero to seven. A total of 800 vegetation data points were 
collected, 400 for each terrain sample. These observations represent the © 
entire vegetation code population for each selected one kilometer; the same 
one kilometer square was chosen from each database. The question to be 
answered was not whether the two populations were different, the known 
changes ensured that they were different. But, did the changes that were 
required to make the modified one kilometer square terrain more accurately 
reflect the actual terrain result in differences more substantial than 
would be expected between two one kilometer squares chosen at random. 
Parametric, nonparametric and graphical statistical analysis techniques 
were used to compare and contrast the vegetation code distributions. The 
analytical software packages used in this thesis were the statistical 
package SPSS and A Graphical Statistical System (AGSS). AGSS is available 
for use at the Naval Postgraduate School (NPS) under a test site agreement 


with IBM Research. We are indebted to Dr. Peter Welch for making this possible. 
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First, graghical tests for normality were conducted on the vegetation 
code data from each sample to determine if parametric techniques were 
appropriate. As can be seen in the following section, neither vegetation 
code distribution is well approximated by the normal distribution. 
Therefore, nonparametric and graphical tools were used in the analysis. 
Note that it is acceptable to use nonparametric techniques for data that 
does follow the normal distribution. Nonparametric procedures perform 
almost as well as the ¢ and F tests for normal data , and will often 
perform much better under non-normal conditions [DEVO 87}. Graphical tools 
were used to provide visual insights into the data distributions. 

Next, three scenarios fromthe Initial Operational Test and Evaluation 
(IOTE) for the Javelin antitank weapon were run on each the default and 
modified terrain files. The scenarios: a night deliberate defense, a day 
hasty defense and a day deliberate defense were set up to be run on the 
Janus(A) simulation model by CPT Mick McGuire for his thesis The Javelin 
Versus the Dragon II, A Comparative Analysis. These scenarios were 
imported from CPT McGuire's database in a cooperative effort to improve 
the Javelin IOTE. The three scenarios were each run eight times on each 
terrain file for a total of 48 runs. The measure of effectiveness (MOE) 
was the detection range of an enemy target by either force. This MOE was 
used because detection is greatly effected by the vegetation represented 


in the terrain Gatabase. 


B. ANALYSIS OF TERRAIN VEGETATION 

The first test on the terrain samples was atest to determine if the 
terrain sample vegetation codes were from the normal distribution. Normal 
probability plots and detrended normal plots were was used to determine if 
nonparametric or parametric statistical procedures should be used to 
analyze the data. The normal probability plot pairs each observed value 
with its expected value from the normal distribution. If the sample is 


from the normal distribution the points should fall more or less on a 
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straight line [{MEND 90). Figure 6 shows that the points do not cluster 
around a straight line and in Figure 7 shows that the deviations from the 


straight line, the detrended residuals, are not randomly distributed around 


zero. 
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Figure 6 Default Normal Plot 





Detrended Normal Plot of VEGETATION CODE 
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Figure 7 Default Detrended Normal Plot 





Likewise, the data points for the Janus(A) modified terrain do not 


fall along the straight line in Figure 8, and the detrended normal plot, 


24 


Figure 9, shows a lack of randomness in the dispersion of the residuals. 


Normal Plot of VEGETATION CODE 
for JANUS(A) MODIFIED TERRAIN 
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Figure 8 Modified Normal Plot 


Detrended Normal Plot of VEGETATION CODE 
for JANUS(A) MODIFIED TERRAIN 
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Figure 9 Modified Detrended Normal Plot 





These four plots indicate that the vegetation code data in the default 
and modified Janus(A) terrain do not follow the normal distribution. 
Therefore, nonparametric and graphical techniques will be used as the 
analysis tools. By looking at the graphs it cannot easily be determined 


that 400 data points per terrain sample were used in this analysis process. 
} 
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| This is because the vegetation codes can only take the value of discrete 
integers from zero to seven. Therefore, the data for both terrain samples 
was plotted using a bar graph. Figure 10 graphically confirms that the 
vegetation codes do not follow the normal distribution, and further 


indicates that their distributions are not similar. 
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Figure 10 Comparison of Vegetation Codes 





In sum, the preceding five graphs indicated that the vegetation codes 
for the default and modified terrain did not follow the normal 
distribution. Therefore, the assumption of normality could not be made and 
nonparametric and graphical techniques were used to analyze whether the 
vegetation codes in the two terrains files were similar. The Mann-Whitney 
test was used to test the null hypothesis that the population relative 
frequency distributions for the two terrain samples are identical [MEND 
S30 )c The least restrictive alternative is that one distribution is 
stochastically larger than the other [GIBB 92}. The Mann-Whitney test 
requires the assumption of two independent random samples from continuous 
Gistributions. The vegetation code samples clearly do not satisfy that 
assumption. However, the test results do answer the question; “Would two 


independent random samples from the same vegetation code distribution be 
2 
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expected to exhibit differences similar to those between the default and 
modified terrain samples?" The answer is clearly no. The following table 


shows the results of the Mann-Whitney test. 


Table II VEGETATION CODE MANN-WHITNEY TEST 
lann-Whitney U Test = 


VEGCODE 
by TERRAIN 


Mean Rank 
3.76.4 1 TERRAIN Modified 
424.59 TERRAIN Default 
Total 
Corrected for ties 


W Zz 2-Tailed 
150565.0 -3.1360 -0017 





After the vegetation codes are merged and ranked, the U statistic 
represents the number of times a modified terrain vegetation code value 
preceeds a value for the default terrain vegetation code. The very large 
value of U indicates there is a separation of the ordered modified and 
default vegetation code observations and indicates a stochastic difference 
between the distributions. Since the significance level is small (less 
than .025, for a two tailed test) the null hypothesis that the two 
vegetation code distributions are the same is rejected. It is concluded 
that the vegetation codes in the Janus(A) modified terrain are 
stochastically different from the vegetation codes in the Janus(A) default 
terrain. 

The distrbutions of the two vegetation code samples are not known, but 
the following graph, Figure 11, provides some detailed comparisons of the 
distributions of the two data sets. The graph is an empirical quantile- 


quantile plot and is constructed by plotting the quantiles of one empirical 
Uf 
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Gistribution against the corresponding quantiles of the other [CHAM 83]. 
If the distributions were identical, all the points would lie on the line 
y=xX. The points do not lie onthe y = X line, thus the distributions are 
not identical. Because most of the points lie below the y = x line, it is 
clear that the Janus(A) default terrain has the higher vegetation density 
codes. Also, because the data points are discrete integers they form a 


stepwise linear curve. 


O-Q PLOT COMPARISON OF VEGETATION CODES FOR THE 
JANUS(A) DEFAULT AND MODIFIED TERRAIN 
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Figure 11 Vegetation Code 
Quantile-Quantile Plot 





Cc. ANALYSIS OF THE SCENARIO RUNS 

It has been determined that the vegetation codes in the two terrain 
files are stochastically different. Intuition indicates that if the same 
scenario waS run on each terrain file, the detection ranges should also 
come from different distributions. This section will explore this 
hypothesis. In this analysis, vegetation code distribution will be treated 
as a factor at two levels: default and modified. 

The scenarios consisted of a U.S. light infantry company team - BLUE 
force and a North Korean mechanized platoon - RED force. The light 


infantry team is equipped with M16 rifles, a 60mm mortar section, squad 


, 
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automatic weapons (SAW), M60 machine guns, light antitank weapons (LAW) and 
Javelin medium antitank weapons. The company team is also augmented with 
a heavy mortar section. The North Korean platoon is organized with three 
T72 tanks, three BMPs and a heavy mortar section. The BLUE and RED forces 
have 119 and 41 personnel, respectively. The tactical employment of the 
forces follows the known warfighting doctrine of each country. 

To prevent a learning effect when running the scenarios, each run was 
conducted using the Automatic Janus(A) mode. In this mode, the computer 
runs the scenario on saved puck movements (player actions) which were made 
by the initial man in the loop interactive battle. The probabilistic 
outcomes (detection, kill, etc.) are not necessarily the same, so the data 


can be viewed as independent, random samples. 


a The Night Deliberate Defense 
The first scenario was the night deliberate defense. In the 
terrain area where this battle took place, there were many modifications 
to the vegetation. These differences were captured in the following 


descriptive summary. 


Table III NIGHT DELIBERATE DEFENSE 
DETECTION RANGE SUMMARY 


| DEFAULT MODIFIED 
B52 
.03 
aoe 
sien 





4.32 
vis 


Next the Mann - Whitney test was used to test the null hypothesis 
that the population relative frequency distributions for the two detection 
range samples are identical. The significance level was .0004; therefore, 
the null hypothesis is rejected. The population relative frequency 
distributions for the detection ranges in the night deliberate defense for 


the two terrain files were significantly different. 


a) 


Table IV NIGHT DELIBERATE DEFENSE MANN-WHITNEY TEST 


_— 





[Mann-Whitney U Test 
DETECTRG 
by TERRAIN 
Mean Rank 


882.43 TERRAIN Default 
197.62 TERRAIN Modified 


Total 


Corrected for ties 
U W Zz 2-Tailed P 
305710-5 630939.5 -3.5601 .0004 


A graphical depiction showing the differences between the two 


detection range samples is shown in Figures 12 and 13 on the following 


page. 
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Detection Range for Night Deliberate Defense 
(Frequency vs. Detection Range) 
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Figure 12 Night Deliberate Defense Detection Ranges 
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Figure 13 Night Deliberate Defense Detection Ranges 


The quantile-quantile plot for the night deliberate defense, 
Figure 14, shows that the distributions of the sample detection ranges for 


the default terrain and the modified terrain are not the same. Below 2.75 


kilometers the default terrain detection ranges were greater. 


Interestingly, the detection ranges for the modified terrain are similar 


Syl 


to the detection ranges for the default terrain for distances beyond 2.75 
kilometers. This could be explained by a similarity in the terrain 
vegetation codes for the particular areas modeled and the use of infared 


sensors to detect targets at these ranges. 


Q-Q PLOT COMPARISON OF DETECTION RANGES FOR THE 
NIGHT DELIBERATE DEFENSE 
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Figure 14 Detection Range 
Quantile-Quantile Plot 


a The Day Hasty Defense 
The second scenario was the day hasty defense. The terrain in 
this area also required numerous modifications to the vegetation. These 


differences were captured in the following descriptive summary. 


Table V DAY HASTY DEFENSE DETECTION 
RANGE SUMMARY 
DEFAULT MODIFIED 
| Loy 
03 
1206 
= 10) 





2) 
1306 


Again the Mann - Whitney test was used to test the null 


hypothesis that the sample population relative frequency distributions for 


} 


SZ 


the two detection range samples are identical. The significance level was 
.0000. The population relative frequency distributions for the detection 
ranges in the day hasty defense for the two terrain files were 


Significantly different. 


Table VI DAY HASTY DEFENSE MANN-WHITNEY TEST 
Mann-Whitney U Test 


DETECTRG 
by TERRAIN 


Mean Rank 


1396.58 TERRAIN Default 
930.21 TERRAIN Modified 


Total 
Corrected for ties 


VU W Z 2-Tailed P 
361385.0 1319770.0 -16.8017 -0000 





The histograms on the following page graphically show the 


@ifferences between the two detection range samples, Figures 15 and 16. 
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Detection Range for Day Hasty Defense 
(Frequency vs. Detection Range) 
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Figure 15 Day Hasty Defense Detection Ranges 





Detection Range for Day Hasty Defense 
(Frequency vs. Detection Range} 
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Figure 16 Day Hasty Defense Detection Ranges 





The quantile-quantile plot, Figure 17, for the day hasty defense 
shows that the default terrain sample detection ranges are much greater 
than those for the modified terrain. The detection range distributions for 


the two terrains are clearly not identical. 
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Q—Q PLOT COMPARISON OF DETECTION RANGES FOR THE 
DAY HASTY DEFENSE 





Figure 17 Detection Range 
Quantile-Quantile Plot 


oe The Day Deliberate Defense 
In the last scenario, the majority of the engagement area was in 
an open area. Intuitively the modifications to the terrain should have 
little effect on the detection ranges for the two samples. The following 
descriptive summary indicates the similarities between the two detection 


range samples. 


Table VII DAY DELIBERATE DEFENSE 
DETECTION RANGE SUMMARY 





The Mann - Whitney test showed a Significance level of .2267. 
The null hypothesis that the population relative frequency distributions 


for the two detection range samples are identical is not rejected. 
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Table VIII DAY DELIBERATE DEFENSE MANN-WHITNEY TEST 
ftann-Whitney U Test ae = = 


DETECTRG 
by TERRAIN 





Mean Rank Cases 


838.28 658 TERRAIN Default 
809.43 983 TERRAIN Modified 


1641 Total 


Corrected for ties 
W Z 2-Tailed P 


312034.0 Solo le20 =-1.2089 -2267 


U 


The following two histograms also show the distinct similarities 


between the detection range populations for the two samples, Figures 18 and 


ioe 
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Detection Range for Day Deliberate Defense 
(Frequency vs. Detection Range) 


Std. Dev = .95 
a Sty UP eet TE Mean = 1.69 
He Cn Gece Hite a Gite Gee ee: N = 658.00 


DETECTRG 
Janus(A) Default Terrain 


Figure 18 Day Deliberate Defense Detection Ranges 





Detection Range for Day Deliberate Defense 
(Frequency vs. Detection Range) 
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Figure 19 Day Deliberate Defense Detection Ranges 


In the day deliberate defense scenario the distribution of the 
sample detection ranges for the default and modified terrain are not 
identical. However, they are similar (Figure 20), and an assumption that 


the distributions are equal is reasonable. 


Si 


O-Q PLOT COMPARISON OF DETECTION RANGES FOR THE 
DAY DELIBERATE DEFENSE 





Figure 20 Detection Range 
Quantile-Quantile Plot 


4. Aggregated Detection Ranges 

The final analysis was conducted on the aggregated detection 
ranges for the scenarios on the default and modified terrain files. After 
conducting analysis on each scenario and showing that they are different, 
it seems to be wrong to aggregate the data. An analysis of the aggregated 
data was conducted because the scenarios of an IOTE are part of a larger 
unit operation. These operations are usually 72 or 96 hours and are 
composed of any number of scenarios. Thus, the aggregated data provides 
the proponency agency insights into the distribution of the entire 
operational scenario. 

The results of the analysis showed the significance level to be 
.0000. The population relative frequency distrbutions of the two detection 


ranges are not identical. 
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Table IX AGGREGATED DETECTION RANGE MANN-WHITNEY TEST 
U Test ———$—— 


DETECTRG 
by FILTER_$ Filter Status 





Mean Rank Cases 


B25 de. 463 2318 FILTER_$ Default Terrain 
2514.56 3241 FILTER_S$ Modified Terrain 


5559. Total 


Corrected for ties 


U W Z 2-Tailed P 
2896033.0 7304326.0 -14.5814 -0000 


Figures 21 and 22 on the following page also show the differences 


in the two detection samples. 
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Histogram of Detection Range for Janus (A) 
Default Terrain 


(Frequency vs. Detection Range) 
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Histogram of Detection Ranges for Janus (A) 
Modified Terrain 


(Frequency vs. Detection Range) 
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Figure 22 Modified Terrain Detection Ranges 


The quantile-quantile plot of the aggregated detection ranges for 
the two terrain files, Figure 23, shows that their distributions are not 
Similar. At ranges greater than than 2.8 kilometers the sample detection 
range distributions are not identical but they are similar. Below 2.8 


kilometers the detection ranges are greater for the default terrain. 
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Q-Q PLOT COMPARISON OF THE AGGREGATED DETECTION 
RANGES FOR THE DEFAULT AND MODWIED TERRAIN 
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Figure 23 Detection Range 
Quantile-Quantile Plot 


D. ANALYSIS SUMMARY OF THE SCENARIOS 

The analysis indicated that the vegetation code distributions for the 
default terrain and the modified terrain are different. Also the detection 
range distributions for the two terrain files in each scenario were not 
identical. In all but one case, the analysis indicated that the terrain 
had a significant effect on the detection ranges. The day deliberate 
defense was conducted in an area where only minimal terrain modifications 
were needed. This area was virtually devoid of vegetation. Therefore, the 
lack of any statistical difference between the modified terrain and default 
terrain detection ranges was expected. The detection range distributions 
for the day deliberate defense could be assumed to be the same because of 


their similarities. 
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V. CONCLUSIONS /RECOMMENDATIONS 


A. SUMMARY OF FINDINGS 

The purpose of this thesis was to compare and analyze the effects of the 
Janus(A) default terrain and the Janus(A) modified terrain on the simulated 
Javelin antitank weapon operational test. The results suggested that modifying 
the vegetation codes to better represent the actual test site terrain will result 
in a significant difference in the detection ranges. This means that the lines 
of sight (LOS) for direct fire combat systems when employed on the modified 
terrain will be significantly different than those employed on the default 
terrain. 

This analysis does not invalidate the Janus(A) model using the default 
terrain. However, this analyst believes better results could be obtained by 


improving the current terrain database. 


B. RECOMMENDATIONS FOR FURTHER RESEARCH 
The findings of this thesis suggest the following recommendations for 


further research on the Janus(A) terrain database: 


e There seems to be a need for a higher resolution terrain database to capture 
these terrain features. The fifty meter resolution does not provide 
accurate enough detail to represent vegetation, small rolling hills or other 
obstacles that effect the LOS of the model. 


« Research needs to be conducted to more accurately represent the size and 
shape of a particular type of cultural feature in a geographic region (i.e., 
coniferous trees, deciduous trees, orchards, rock croppings, etc). This 
modeling effort should also look to represent these features from a ground 
level view as well as from an aerial view. The Perspective View Generator 
and Analysis Systems for Unmanned Sensors (PEGASUS) replicator has made in- 
roads in this area. 
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APPENDIX A: PRIMARY FACTORS 


These are the primary factors associated with each basic terrain 
factor used in the Waterways Experimentation Station (WES) process. 


Land use 
General land use 
Forest stands 
Agricultural crops 
Understory density 
Urban structures 
Embankments 
Slope 
Slope, percent 
Azimuth, degree 
Elevation, meter 
Soils 
Soil type 
Depth of soil 
Obstacles 
General vertical obstacles 
Embankments 
Roadside ditches 
Linear features 
Roads 
Rivers and streams 
Embankments 
Roadside ditches 
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APPENDIX B: JANUS(A) DEFAULT AND MODIFIED SCENARIO TERRAIN 


The first picture shows the Janus(A) default terrain for the Javelin 
test area. As can be seen, the vegetation areas have a homogeneous density 
over a particular area. The second picture shows the Janus(A) modified 
terrain. There is a marked difference in the densities of vegetation areas 
compared to the default terrain. Both sets of terrain have a resolution 
of 50 meters and a contour interval of 33 feet. 
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